Signal processing apparatus

ABSTRACT

An image signal processing apparatus for quantizing an inputted moving image signal according to a quantization step so that a code amount of one frame becomes a target code amount, and for variable-length-coding the quantized moving image signal, in which a minimum value of the quantization step is determined for each frame, and the quantization step is determined so as not to be less than the minimum value.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a signal processing apparatus, and inparticular, to coding process of an image signal.

2. Related Background Art

The MPEG standard is known as an image signal compression and codingtechnology.

In a coding circuit for coding the image signal and thereby generatingdata in accordance with the MPEG standard, a target code amount isdetermined for each picture based on a target data rate. And based onthe target code amount of each picture, the target code amount isassigned equally to all the macro blocks constituting each picture.

Thereafter, a quantization step for coding each macro block isdetermined in order to obtain the target code amount, and then coding isperformed.

Thus, if the quantization step is set by establishing the target codeamount equally to each macro block, the following problems may beraised.

For instance, in the case where there are a macro block including analmost motionless image and a macro block including an image havinglarge motion in one P or B picture, almost no difference data can beobtained from the macro block of the almost motionless image. For thatreason, the quantization step for attaining the target code amount isset to be small.

Inversely, as for the macro block of the image large motion, the valueof difference data is large so that the quantization step must be set tobe large in order to render the code amount of the macro block smallerthan the target code amount.

However, as for the almost motionless image, even if it is coded byrendering the quantization step smaller than necessary, this codingresults in increase of the code amount, while this increase does notresult in significant effect visually. On the other hand, as for theimage having large motion, setting the quantization step to be smallwill result in significant effect visually. Nevertheless, thequantization step cannot be set to be small since such the smallerquantization step results in code amount that exceeds the target codeamount set equally for each macro block.

Thus, the target code amount is assigned equally to each macro block inthe prior art, and so it is impossible to assign a large code amount toa. portion which is less effective visually and assign a necessary codeamount to a portion which is more effective visually.

SUMMARY OF THE INVENTION

An object of the present invention is to solve these problems.

Another object of the present invention is to implement assignment of acode amount without any waste.

A further object of the present invention is to determine an optimumquantization step considering visual effects.

In order to solve these problems, according to a scope of the presentinvention, an image signal processing apparatus of the present inventioncomprises: quantization means for quantizing an inputted moving imagesignal according to a quantization step; coding means forvariable-length-coding the moving image signal quantized by thequantization means; and control means for determining a minimum value ofthe quantization step in predetermined units and determining thequantization step so as not to be less than the minimum value.

Any object and characteristic of the present invention other than thosedescribed above will be clarified by the following detailed descriptionof the embodiments of the present invention referring to the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a configuration of a coding unit to whichthe present invention is applied;

FIG. 2 is a diagram showing a configuration of an image pickup apparatusto which the present invention is applied;

FIG. 3 is a diagram showing an appearance of order of image dataaccompanying coding process;

FIG. 4 is a diagram showing predictive coding operation; and

FIG. 5 is a diagram showing an appearance of a quantization step in oneframe.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereafter, embodiments of the present invention will be described byusing the drawings.

FIG. 2 is a block diagram showing a configuration of an image pickupapparatus to which the present invention is applied.

In FIG. 2, an image pickup unit 201 includes an optical system such as alens and its drive mechanism, an image pickup element such as a CCD, anda conventional camera signal processing circuit and so on. A coding unit202 codes a moving image signal outputted from the image pickup unit 201in accordance with the MPEG method using motion compensation predictivecoding, and outputs the coded signal to a recording unit 203. Therecording unit 203 adds an error correction check code, synchronizationdata and other additional data to the coded moving image signal from thecoding unit 202 to convert it into a form of a recording format, andrecords the converted signal on a magneto-optical disk 204 with aconventional magneto-optical recording method.

In addition, a control unit 205 controls operation of each unit of theapparatus according to an instruction by a user's operation of anoperation unit 206.

Next, the coding unit 202 which is a characteristic configuration ofthis embodiment will be described.

FIG. 1 is a diagram showing a configuration of the coding unit 202.

The coding unit 202 in FIG. 1 performs coding process in accordance withthe MPEG2 method using motion compensation predictive coding, aspreviously mentioned. In the MPEG2 method, coding is performed byselectively using intra-coding and inter-coding for each frame. Theintra-coding is a coding method of performing the coding only by usingdata in the same frame, and the inter-coding is a coding method ofperforming the coding by using a plurality of frames.

In addition, the MPEG2 provides three picture types, I picture forcoding all the data of one frame by the intra-coding, P picture forperforming predictive coding by using a preceding frame and B picturefor performing predictive coding by using preceding and subsequentframes. And a set of frames from one I picture to a frame immediatelybefore the next I picture is called GOP (Group Of Pictures), which canbe handled as one coding unit.

In FIG. 1, the moving image signal outputted from the image pickup unit201 is outputted to a picture rearrangement circuit 102 via an inputterminal 101. The picture rearrangement circuit 102 has a memory capableof storing the moving image signals of a plurality of frames, andchanges order of frames of the inputted moving image signals to theorder adequate for coding and outputs the rearranged signals.

The operation of the picture rearrangement circuit 102 will be describedby using FIG. 3.

Reference numeral 301 in FIG. 3 denotes the order of frames of themoving image signals to be inputted to the picture rearrangement circuit102, which signals inputted in order of a first frame, a second frame, athird frame . . . . Reference numeral 302 denotes the order of themoving image data outputted from the picture rearrangement circuit 102,which outputs the signals in order of the third frame, first frame,second frame . . .

FIG. 4 is a diagram showing an appearance of the coding of the movingimage signals thus outputted as I picture, P picture and B picture.

Numerals denoted by a reference numeral 401 in FIG. 4 corresponds to theframe numbers in FIG. 3, and 402 denotes the picture types of eachframe. In FIG. 4, 15 frames constitute one GOP, and the third frame isthe I picture at the head. The first and second frames are the Bpictures of the preceding GOP, and as shown in FIG. 4, the first andsecond frames are coded by using an image signal of the third framewhich is the I picture and the image signal of the P picture of theimmediately preceding GOP. In addition, since the third frame is the Ipicture, it is only coded with the image data of the third frame. Inaddition, the sixth frame is the P picture, and forward predictivecoding is performed thereto by using the third frame which is theimmediately preceding I picture. Hereafter, the image signals of the Bpicture and P picture are coded likewise respectively.

In order to implement such coding process, the image signal outputtedfrom the picture rearrangement circuit 102 in the order of FIG. 3 orFIG. 4 is outputted to a switch 103, a subtractor 104 and a motioncompensation predictive coding circuit 111.

In the case where the image signal of the I picture is outputted, theswitch 103 is connected to a terminal A, and the image signal outputtedfrom the picture rearrangement circuit 102 is outputted as it is to aDCT circuit 105.

In addition, in the case of the P picture or B picture, the switch 103is connected to a terminal B side. The subtractor 104 obtains thedifference between the image signal of the P picture or B pictureoutputted from the picture rearrangement circuit 102 and a predictiveimage signal from the motion compensation predictive circuit 111, andoutputs it to the DCT circuit 105 via the switch 103. Thus, redundancyin a time-axis direction is reduced.

The DCT circuit 105 DCT-processes the image data of the I picture ordifference data of the P picture or B picture outputted from the switch103, and outputs the processed data to a quantization circuit 106. Thequantization circuit 106 quantizes a DCT coefficient of each pictureoutputted from the DCT circuit 105 in accordance with a quantizationstep indicated by a quantization step calculation circuit 115 asmentioned later, and outputs the quantized data to an inversequantization circuit 107 and a variable-length coding circuit 112.

The variable-length coding circuit 112 variable-length-codes quantizedimage data, and outputs the coded data to a buffer memory 113. The imagedata stored in the buffer memory 113 is read in predetermined timing,and is outputted to the recording unit 203 via an output terminal 116.

On the other hand, the inverse quantization circuit 107 inverselyquantizes the image signal from the quantization circuit 106, andoutputs the inverse-quantized signal to an inverse DCT circuit 108. Theinverse DCT circuit 108 inversely DCT-processes the inversely quantizedimage signal and outputs the processed signal to an adder 109.

Here, in the case where the data from the inverse DCT circuit 108 isthat of the I picture, a switch 110 is turned off, and the image dataoutputted from the inverse DCT circuit 108 passes through the adder 109as it is to be outputted to the motion compensation predictive circuit111. In addition, in the case where the data from the inverse DCTcircuit 108 is the P picture or B picture, the switch 110 is turned on.And the predictive image signal outputted to the subtractor 104 and thedata (local decode data) from the inverse DCT circuit 108 are added bythe adder 109 to be outputted to the motion compensation predictivecircuit 111.

The motion compensation predictive circuit 111 includes a memory forstoring the locally-decoded data of the I picture and P pictureoutputted from the adder 109, and divides the data of the P picture or Bpicture into a plurality of macro blocks comprised of a plurality ofpixels respectively and reads them into a macro block valley. And itcompares the data of the P picture or B picture outputted from thepicture rearrangement circuit 102 to the image data of a reference framestored in the memory in the units of a macro block so as to detect thepredictive image signal of a smaller difference and a motion vectorthereof. And it outputs the predictive image signal to the subtractor104, and also outputs the data of the motion vector to thevariable-length coding circuit 112. The variable-length coding circuit112 also codes the data of the motion vector, and outputs the codedmotion vector together with the image data to the buffer memory 113.

A rate control circuit 114 monitors the amount of the coded image datastored in the buffer memory 113, to inform the quantization step controlcircuit 115 thereof. The quantization step control circuit 115determines a target code amount for each picture so as to attain aspecified target data rate, based on the amount of the image data storedin the buffer memory 113 and the information on the picture typeindicated by the picture rearrangement circuit 102.

Moreover, the target data rate may be automatically set by theapparatus, or it is also possible for the user to set an arbitrary valueby operating the operation unit 206.

And in this embodiment, a minimum value of the quantization step foreach picture is determined according to a specified target data rate. Tobe more specific, the quantization step control circuit 115 hasquantization tables each of which includes different quantization stepscorresponding to different numbers, and of the quantization tables,those having the quantization steps corresponding to the number smallerthan the number established as the minimum value thereof are not used.For instance, in the case where the target data rate is 6 Mbps, theminimum value of the quantization step of the I picture is 8, that ofthe P picture is 9, and that of the B picture is 12.

And the quantization step is calculated by assigning the target codeamount equally to each macro block. In this case, however, thequantization step of each macro block is set so that the quantizationstep will not be less than the minimum value set for each picture. As aresult of thus setting the quantization step, codes of a macro blockwhose code amount does not reach the target code amount is assigned toanother macro block of which quantization step is not less than theminimum value.

And the quantization step control circuit 115 outputs to thequantization circuit 106 the quantization step of each macro block whichis finally set. It also outputs the information indicating thequantization step of each macro block to the recording unit 203 in FIG.2. The recording unit 203 adds the information indicating thequantization step of each macro block to the coded image data to berecorded together.

FIG. 5 is a diagram showing an example of the quantization step of alarge number of macro blocks in one frame.

Reference numeral 501 denotes a diagram showing an appearance of thequantization step of each macro block in the case of setting no minimumvalue of the quantization step, and 502 denotes a diagram showing anappearance of the quantization step of each macro block in thisembodiment with the minimum value of the quantization step set.

In the diagram 501, the quantization steps of the macro blocks on theupside of a picture are set to be relatively small such as 6 to 8, andthose of the macro blocks on the downside of the picture are set to berelatively large such as 14 to 20. For this reason, it is consideredthat the image quantized in the quantization step in FIG. 5 hasrelatively less motion on the upside of the screen, and has a relativelylarger motion on the downside of the screen.

Here, in case of setting the minimum value of the quantization step at10, the quantization steps of the macro blocks on the upside of thescreen become approximately 10 as shown in the diagram 502. And extracode amounts generated as a result of rendering the quantization stepslarger than in the diagram 501 are assigned to the macro blocks on thedownside of the screen so that the quantization steps of the macroblocks on the downside of the diagram 502 can be rendered smaller thanin the diagram 501 such as 12 to 14 and a higher-resolution image can beobtained. Moreover, even in the case where the quantization steps arethus changed, there is no difference in a data amount per one picture.

Thus, according to this embodiment, the minimum value of thequantization step is set for each picture type, and the quantizationstep of each macro block is set not to be less than the minimum value,so that it is possible to avoid assigning a large code amount to themacro blocks whose visual effects cannot be expected even if thequantization step is rendered small.

Next, a second embodiment will be described. In this embodiment, appliedconfigurations are the same as those in FIGS. 1 and 2.

While the target code amount of each picture is assigned based on thespecified target data rate in the first embodiment, the minimum value ofthe quantization step is further set for each picture according to acharacteristic of input image data in this embodiment.

To be more specific, a degree of the motion of the input image data isdetected from results of comparing the data between the frames of theimage data inputted by the picture rearrangement circuit 102 or fromtotaling values of the motion vectors of one frame detected by themotion compensation predictive circuit 111.

In the case of less motion, it is considered that the difference valuefrom the predictive image signal becomes smaller as to the data of the Ppicture or B picture, and so the minimum value of the quantization stepof the I picture is set to be smaller, and the minimum values of thequantization steps of the P picture and B picture are set to be larger.

In addition, in the case of larger motion with random movement of theimage, the minimum value of the quantization step of the I picture isestablished not to be different from those of the P picture and Bpicture.

Thus, it is possible, by changing the minimum value of the quantizationstep of each picture according to the characteristic of the input image,to effectively assign the code amount according to the characteristic ofthe image.

Moreover, while the characteristic of the input image data is detectedto determine the degree of motion, based on the results of comparing theimage data between the frames by the picture rearrangement circuit 102or totaling the motion vectors in the second embodiment, it is alsopossible, by using a vibration correcting function of the image pickupunit 201 in FIG. 2 for instance, to attain a configuration wherein thedata showing vibration amount obtained from an angular speed sensor orimage processing is inputted from the image pickup unit 201 and themotion of the input image data is determined based thereon.

Furthermore, it is also possible to determine the motion based on anoperation signal of a zoom key by the operation unit 206.

To be more specific, in the case where zoom speed is slow, it isconsidered that the motion may be detected between the frames but thedifference value from the predictive image signal also becomes smaller.Therefore, as in the case of less motion, the minimum value of thequantization step of the I picture should be set to be smaller, and theminimum value of the quantization steps of the P picture and B pictureshould be set to be larger.

In addition, while the cases of applying the present invention to theimage pickup apparatus are described in the aforementioned embodiments,it is also possible to apply the present invention otherwise to thecases of quantizing and coding the image signal with the same effects.

In addition, it is also possible to constitute all or a part of theconfigurations of the coding unit in FIG. 1 as one IC chip, andfurthermore, it is also possible to implement the functions in FIG. 1with software processing using a microprocessor, a register, a RAM andso on.

In this case, a storage medium such as the memory storing a program forattaining the functions in FIG. 1 also constitutes the presentinvention.

As described above, according to the present invention, it is possibleto implement assignment of the code amount without any waste and todetermine an optimum quantization step considering visual effects.

Many widely different embodiments of the present invention may beconstructed without departing from the spirit and scope of the presentinvention. It should be understood that the present invention is notlimited to the specific embodiments described in the specification,except as defined in the appended claims.

1.-22. (canceled)
 23. An image signal processing apparatus comprising: arearranging unit configured to rearrange pictures of an input movingimage signal in an order of predetermined picture types; a quantizationunit configured to set a quantization step, and to quantize each pictureof the moving image signal output from said rearranging unit accordingto the set quantization step; a coding unit configured tovariable-length-encode the moving image signal quantized by saidquantization unit; and a quantization step control unit having aquantization table in which values corresponding to a plurality ofquantization steps to be used by said quantization unit are described,said quantization step control unit being configured to effectquantization step control to fixedly establish a different minimum valueof the quantization steps in accordance with a picture type of thepicture rearranged by said rearranging unit, so as not to set aquantization step corresponding to a value smaller than the minimumvalue which is established to the rearranged picture, to saidquantization unit.
 24. An apparatus according to claim 23, wherein saidquantization step control unit effects control so as to assign an amountof redundant codes generated in the rearranged picture due to fixedlyestablishing of the minimum value of the quantization step, to a lowerarea of the rearrangement picture.
 25. An apparatus according to claim23, wherein said quantization step control unit fixedly establishes theminimum value of the quantization step in such a manner that the smallerminimum value is established for the I picture type than the P and Bpicture types.
 26. An apparatus according to claim 23, wherein saidquantization step control unit establishes the minimum value of thequantization value based on a code amount of a coded moving image signaloutputted from said coding unit in a predetermined period.
 27. Anapparatus according to claim 23, wherein said quantization step controlunit establishes the minimum value of the quantization value based on acharacteristic of the input moving image signal.
 28. An apparatusaccording to claim 23, wherein said quantization step control unitestablishes the minimum value of the quantization value based on motionof the input moving image signal.
 29. An apparatus according to claim28, further comprising a motion compensation predictive unit configuredto generate a predictive image signal from a reference picture of thepicture to be coded of the moving image signal, and detect a motionvector for the predictive image signal, wherein said quantization stepcontrol unit detects the motion of the input moving image signal basedon the motion vector detected by said motion compensation predictiveunit.
 30. An apparatus according to claim 28, further comprising animage pickup unit configured to generate the input moving image signal,wherein said quantization step control unit further establishes theminimum value of the quantization value according to an operating stateof said image pickup unit.
 31. An image signal processing method,comprising the steps of: rearranging pictures of an input moving imagesignal in an order of predetermined picture types; setting aquantization step, and quantizing each picture of the moving imagesignal output in said rearranging step according to the set quantizationstep; variable-length encoding the moving image signal quantized in saidquantization step; and effecting quantization step control in accordancewith a quantization table in which values corresponding to a pluralityof quantization steps to be used in said quantization step aredescribed, to fixedly establish a different minimum value of thequantization steps in accordance with a type of the picture rearrangedin said rearranging step so as not to set a quantization stepcorresponding to a value smaller than the minimum value which isestablished the rearranged picture, to said quantization step.